Distributed computing systems, sometimes also referred to as cloud computing systems, are used to provide services to electronic devices operated by end users. In a cloud computing system, the hardware and software computer resources configuration is hidden from the end user. Cloud computing systems may include servers, network storage devices, routers, gateways, communication links, and other physical hardware and software platforms. However, because services are deployed on physical hardware and software platforms which are hidden from end users, they can be managed, upgraded, replaced or otherwise changed by a system administrator without the end users being aware of or affected by the change.
In existing cloud and other distributed computing systems, the creator of services or the operator of the cloud system must know in advance which applications (or types of applications) will be deployed and provide associated instructions for the number and types of resources that are deployed in the cloud system to support processing of the applications. During operation, a load balancer can operate to direct requests from user electronic devices to particular ones of the resources for processing by the associated applications. Although load balancers can provide better balancing of resource utilization, they may not sufficiently improve the efficiency at which resources are deployed and used, which may have a substantial effect on cost in view of the potential large number of resources that can be deployed in some distributed computer systems.